H.264, also known as Moving Pictures Experts Group-4 (MPEG-4) Part 10 or MPEG-4 Advanced Video Coding (AVC), is a standard for video compression. A video includes a series of pictures (or frames), with each frame consisting of a two-dimensional array of pixels. The pixels are divided into macroblocks (a 16×16 array of pixels). Each macroblock has a macroblock number; in general, the macroblocks are numbered starting at the top-left of the frame, in increasing order from left-to-right and top-to-bottom. The macroblocks can be grouped into slices, and the slices can be grouped into slice groups. Macroblocks within a slice are arranged in ascending order by macroblock number. A slice can include any number of macroblocks, which may or may not be contiguous; that is, macroblocks in one slice may be interspersed among macroblocks of one or more other slices of other slice groups—macroblocks from slices in the same slice group are not interspersed with each other. The feature of H.264 that allows macroblocks to be grouped into slices as just described is generally referred to as flexible macroblock ordering (FMO).
FMO is one of the error resiliency tools that can be used by a decoder to conceal errors if slices are lost or corrupted during transmission. Macroblocks in a missing or corrupted slice can be reconstructed by interpolating or extrapolating macroblock information from another slice. More specifically, a correctly received slice can be decoded, and the information in that slice can be used to derive information for another slice.
Another H.264 feature is generally referred to as arbitrary slice ordering (ASO). With ASO, slices can be transmitted in any order. For example, a slice may be sent as soon as it is ready—that is, a slice may be streamed to a decoder as soon as all of the macroblocks that make up that slice are encoded. As a result, a slice from one slice group may be sent, followed by a slice from another slice group, followed by another slice from the first slice group, and so on.
Unfortunately, ASO places additional burdens on decoders, making their implementation more complicated. In the example of FIG. 1, in order to decode macroblock M1, information may be needed for macroblock M2 (the nearest neighbor on the left of macroblock M1) and perhaps for macroblock M3 (the nearest neighbor above macroblock M1). However, macroblock M1 may be in one slice (e.g., slice S2) and the neighboring macroblocks M2 and M3 may be parts of a different slice (e.g., slice S1) or slices. With ASO enabled, slice S2 may be sent to the decoder before slice S1. Subsequently, macroblock M1 may be ready for decoding, but that decoding may have to be delayed if slice S1 has not yet arrived at the decoder. As a result, slice S2 will be stored at the decoder until slice S1 arrives.
In general, because of ASO, decoders need to have the memory capacity to store slices of encoded data until other slices needed for decoding are available. Furthermore, decoders need to have the processing capability to sort through the stored slices and arrange them as needed for decoding.